Anonymization of data within a streams environment

ABSTRACT

Streams applications may decrypt encrypted data even though the decrypted data is not used by an operator. Operator properties are defined to permit decryption of data within the operator based on a number of criteria. By limiting the number of operators that decrypt encrypted data, the anonymous nature of the data is further preserved. Operator properties also indicate whether an operator should send encrypted or decrypted data to a downstream operator.

BACKGROUND

Embodiments of the present disclosure generally relate to streamcomputing applications using streaming data. Specifically, theembodiments disclose techniques for processing encrypted data withinstreams computing applications that use streaming data.

SUMMARY

Embodiments disclosed herein provide a method, system and computerprogram product for performing an operation, the operation includingproviding a plurality of processing elements comprising one or moreoperators, the operators configured to process streaming data tuples.The operation then defines attributes of the operators, the attributesincluding at least an access indicator, the access indicator definingprocessing rules for tuples containing encrypted data. The operationthen establishes an operator graph of multiple operators, the operatorgraph defining at least one execution path in which a first operator ofthe plurality of operators is configured to receive data tuples from atleast one upstream operator and transmit data tuples to at least onedownstream operator. The operation then, upon receiving a first datastream containing a first tuple containing encrypted data in the firstoperator, determines, based on the access indicator of the firstoperator, whether to decrypt the encrypted data in the first operator.The operation, upon determining that the access indicator of the firstoperator permits decryption of the encrypted data, decrypts theencrypted data. The operation, upon determining that the accessindicator of the first operator does not permit decryption of theencrypted data, does not decrypt the encrypted data. The operation thentransmits the tuple to a second operator, downstream from the firstoperator.

An application may decrypt encrypted data even if the application is notgoing to perform any operations on the decrypted data. In streamsprogramming and data mining applications, the data in its associateddata record, or its associated tuple in the case of streams programming,is the information needed to process the data. For example, it is notnecessary to see social security numbers when trying to determine whichgender is more susceptible to a given medical condition. It is also notnecessary to view a customer's credit card information when determiningthe average number of products purchased in a single customertransaction. Additionally, it is not necessary to expose informationidentifying a customer's bank account when identifying the bank orautomated teller machine the customer uses the most frequently.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited aspects are attained andcan be understood in detail, a more particular description ofembodiments of the disclosure, briefly summarized above, may be had byreference to the appended drawings.

It is to be noted, however, that the appended drawings illustrate onlytypical embodiments of this disclosure and are therefore not to beconsidered limiting of its scope, for the disclosure may admit to otherequally effective embodiments.

FIGS. 1A-1B illustrate a computing infrastructure configured to executea stream computing application, according to embodiments disclosedherein.

FIG. 2 is a more detailed view of the compute node of FIGS. 1A-1B,according to one embodiment disclosed herein.

FIG. 3 is a more detailed view of the server management system of FIGS.1A-1B, according to one embodiment disclosed herein.

FIG. 4 is a flowchart depicting a method for processing encrypted data,according to one embodiment disclosed herein.

FIG. 5 is a flowchart depicting conditions permitting the decryption ofencrypted data, according to one embodiment disclosed herein.

FIG. 6 is a flowchart depicting methods of decrypting encrypted data,according to one embodiment disclosed herein.

FIG. 7 is a flowchart depicting a method for transmitting data,according to one embodiment disclosed herein.

FIG. 8 illustrates a table indicating whether a tuples containingencrypted data will be decrypted within an operator, according to oneembodiment described herein.

DETAILED DESCRIPTION

Embodiments disclosed herein provide a method, system and computerprogram product for performing an operation, the operation includingproviding a plurality of processing elements comprising one or moreoperators, the operators configured to process streaming data tuples.The operation then defines attributes of the operators, the attributesincluding at least an access indicator, the access indicator definingprocessing rules for tuples containing encrypted data. The operationthen establishes an operator graph of multiple operators, the operatorgraph defining at least one execution path in which a first operator ofthe plurality of operators is configured to receive data tuples from atleast one upstream operator and transmit data tuples to at least onedownstream operator. The operation then, upon receiving a first datastream containing a first tuple containing encrypted data in the firstoperator, determines, based on the access indicator of the firstoperator, whether to decrypt the encrypted data in the first operator.The operation, upon determining that the access indicator of the firstoperator permits decryption of the encrypted data, decrypts theencrypted data. The operation, upon determining that the accessindicator of the first operator does not permit decryption of theencrypted data, does not decrypt the data. The operation then transmitsthe tuple to a second operator, downstream from the first operator.

Most applications need access to data and then act upon that data. Dueto many regulations in different industries, some of the data thatapplications traditionally access now needs to be either storedseparately from data with which it traditionally resides, or encryptedsuch that upon accessing the data, keys or some methodology is needed tounlock the underlying data. Many forms of data, such as social securitynumbers, can no longer be stored in traditional database tables withouthaving substantial protection methods in place for accessing the data.

Stream-based computing and stream-based database computing are emergingas a developing technology for database systems. Products are availablewhich allow users to create applications that process and querystreaming data before it reaches a database file. With this emergingtechnology, users can specify processing logic to apply to inbound datarecords while they are “in flight,” with the results available in a veryshort amount of time, often in milliseconds. Constructing an applicationusing this type of processing has opened up a new programming paradigmthat will allow for a broad variety of innovative applications, systemsand processes to be developed, as well as present new challenges forapplication programmers and database developers.

In a stream computing application, operators are connected to oneanother such that data flows from one operator to the next (e.g., over aTCP/IP socket). Scalability is reached by distributing an applicationacross nodes by creating executables (i.e., processing elements), aswell as replicating processing elements on multiple nodes and loadbalancing among them. Operators in a stream computing application can befused together to form a processing element that is executable. Doing soallows processing elements to share a common process space, resulting inmuch faster communication between operators than is available usinginter-process communication techniques (e.g., using a TCP/IP socket).Further, processing elements can be inserted or removed dynamically froman operator graph representing the flow of data through the streamcomputing application.

In streams programming, data is sent in all directions through operatorgraphs. Many operators are granular in nature and perform specifictasks. Some operators may or may not require access to all of theencrypted data, or in some cases, they require access to encrypted data,but the data they generate is not encrypted.

Embodiments described herein disclose a set of mechanisms that allowstreams programming to operate as efficiently as possible on encrypteddata. Operator properties are set for given tuple attributes, or theattributes are set on the tuple itself to decide how to handle encrypteddata. Embodiments described herein disclose decrypting data inside theoperator itself, meaning the data may be shipped to another node toperform the decryption, or the operator and the state of the operatormay be moved to a given node to perform the decryption.

In the following, reference is made to embodiments of the disclosure.However, it should be understood that the disclosure is not limited tospecific described embodiments. Instead, any combination of thefollowing features and elements, whether related to differentembodiments or not, is contemplated to implement and practice thedisclosure. Furthermore, although embodiments of the disclosure mayachieve advantages over other possible solutions and/or over the priorart, whether or not a particular advantage is achieved by a givenembodiment is not limiting of the disclosure. Thus, the followingaspects, features, embodiments and advantages are merely illustrativeand are not considered elements or limitations of the appended claimsexcept where explicitly recited in a claim(s). Likewise, reference to“the invention” shall not be construed as a generalization of anyinventive subject matter disclosed herein and shall not be considered tobe an element or limitation of the appended claims except whereexplicitly recited in a claim(s).

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present disclosure may take theform of an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present disclosure may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including but not limited to wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent disclosure may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present disclosure are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Embodiments of the disclosure may be provided to end users through acloud computing infrastructure. Cloud computing generally refers to theprovision of scalable computing resources as a service over a network.More formally, cloud computing may be defined as a computing capabilitythat provides an abstraction between the computing resource and itsunderlying technical architecture (e.g., servers, storage, networks),enabling convenient, on-demand network access to a shared pool ofconfigurable computing resources that can be rapidly provisioned andreleased with minimal management effort or service provider interaction.Thus, cloud computing allows a user to access virtual computingresources (e.g., storage, data, applications, and even completevirtualized computing systems) in “the cloud,” without regard for theunderlying physical systems (or locations of those systems) used toprovide the computing resources.

Typically, cloud computing resources are provided to a user on apay-per-use basis, where users are charged only for the computingresources actually used (e.g., an amount of storage space used by a useror a number of virtualized systems instantiated by the user). A user canaccess any of the resources that reside in the cloud at any time, andfrom anywhere across the Internet. In context of the present disclosure,a user may access applications or related data available in the cloud.For example, the nodes used to create a stream computing application maybe virtual machines hosted by a cloud service provider. Doing so allowsa user to access this information from any computing system attached toa network connected to the cloud (e.g., the Internet).

FIGS. 1A-1B illustrate a computing infrastructure configured to executea stream computing application, according to one embodiment of thedisclosure. As shown, the computing infrastructure 100 includes amanagement system 105 and a plurality of compute nodes 130 ₁₋₄, eachconnected to a communications network 120. Also, the management system105 includes an operator graph 132 and a stream manager 134. Asdescribed in greater detail below, the operator graph 132 represents astream computing application beginning from one or more operators in oneor more source processing elements (PEs) through to one or moreoperators in one or more sink PEs. This flow from source to sink is alsogenerally referred to herein as an execution path. Generally, dataattributes flow into an operator of a source PE of a stream computingapplication and are processed by that operator. Typically, operatorsreceive an N-tuple of data attributes from the stream as well as emit anN-tuple of data attributes into the stream (except for operators in asink PE where the stream terminates).

In general, a “tuple” is a single instance of a set of data attributesthat follow the formatting of a schema, where the schema establishes aset of typed data attributes that may be used. For example, the tuplemay be a chunk or portion of divisible data such as a data type (e.g.,string, integer, Boolean, etc.) or combination of data types. In oneembodiment, a “tuple” may include one or more attributes with anassigned value—e.g., Tuple 1: {sym=“Fe”, no=26} where “sym” and “no” arepossible attributes in the schema (i.e., a string and integer,respectively) and “Fe” and “26” are the values. In one embodiment, oneor more attributes of a data tuple may be encrypted. However, not alloperators have the correct keys and mechanisms to decrypt data tuples.Based on the access indicators of the operators, the operator may or maynot be able to decrypt data tuples.

Of course, the N-tuple received by an operator need not be the sameN-tuple sent downstream. Additionally, operators could be configured toreceive or emit tuples in other formats (e.g., the PEs or operatorscould exchange data marked up as XML documents). Based on policies andoperator access indicators, operators that have decrypted data tuplesmay reencrypt the decrypted data tuples before sending them downstream,or may emit the decrypted data tuples. Furthermore, each operator withina PE may be configured to carry out any form of data processingfunctions on the received tuple, including, for example, writing todatabase tables or performing other database operations such as datajoins, splits, reads, etc., as well as performing other data analyticfunctions or operations.

The stream manager 134 may be configured to monitor a stream computingapplication running on the compute nodes 130 ₁₋₄, as well as to changethe deployment of the operator graph 132. The stream manager 134 maymove processing elements (PEs) from one compute node 130 to another, forexample, to manage the processing loads of the compute nodes 130 in thecomputing infrastructure 100. Further, stream manager 134 may controlthe stream computing application by inserting, removing, fusing,un-fusing, or otherwise modifying the processing elements and operators(or what data tuples flow to the processing elements and operators)running on the compute nodes 130 ₁₋₄. One example of a stream computingapplication is IBM®'s InfoSphere® Streams (InfoSphere® is a trademark ofInternational Business Machines Corporation, registered in manyjurisdictions worldwide).

FIG. 1B illustrates an example processing element graph that includesten processing elements (labeled as PE1-PE10) running on the computenodes 130 ₁₋₄. A processing element is composed of one or more operatorsfused together into an independently running process with its ownprocess ID (PID) and memory space. In cases where two (or more)processing elements are running independently, inter-processcommunication may occur using a “transport” (e.g., a network socket, aTCP/IP socket, or shared memory). However, when operators are fusedtogether, the fused operators can use more rapid communicationtechniques for passing tuples among operators in each processingelement.

As shown, the processing element graph begins at a source 135 (thatflows into the processing element labeled PE1) and ends at sink 140 ₁₋₂(that flows from the processing elements labeled as PE6 and PE10).Compute node 130 ₁ includes the processing elements PE1, PE2 and PE3.Source 135 flows into operators in the processing element PE1, which inturn emits tuples that are received by PE2 and PE3. For example,operators in PE1 may split data attributes received in a tuple and passsome data attributes to PE2, while passing other data attributes to PE3.Data that flows to PE2 is processed by the operators contained in PE2,and the resulting tuples are then emitted to the operators in PE4 oncompute node 130 ₂. Likewise, the data tuples emitted by the operatorsin PE4 flow to the operators sink PE6 140 ₁. Similarly, data tuplesflowing from operators in PE3 to operators in PE5 also reach operatorsin sink PE6 140 ₁. Thus, in addition to being a sink for this exampleprocessing element graph, operators in PE6 could be configured toperform a join operation, combining tuples received from operators inPE4 and PE5. This example processing element graph also shows datatuples flowing from PE3 to PE7 on compute node 130 ₃, which itself showsdata tuples flowing to operators in PE8 and looping back to operators inPE7. Data tuples emitted from operators in PE8 flow to operators in PE9on compute node 130 ₄, which in turn emits tuples to be processed byoperators in sink PE10 140 ₂.

Because a processing element is a collection of fused operators, it isequally correct to describe the operator graph as execution pathsbetween specific operators, which may include execution paths todifferent operators within the same processing element. FIG. 1Billustrates execution paths between processing elements for the sake ofclarity.

In one embodiment, the stream manger 134 may be able to communicate withother operator graphs executing in a stream computing application. Thatis, the compute nodes 130 may host operator graphs executing inparallel. The stream manager 134 may be able to communicate with astream manager associated with those parallel operator graphs using, forexample, a shared memory where messages and commands may be passed.Alternatively, stream manager 134 may be part of a hierarchicalarrangement of stream managers that allow the different stream managersto communicate. The stream manager 134 may use the manager hierarchy orthe shared memory to instruct a different stream manager to optimize anoperator graph in the stream computing application that is sharing thesame compute nodes 130 (i.e., hardware resources) as the operator graphshown in FIG. 1B. Additionally, the hierarchical arrangement may managestream managers across different compute nodes, for example, a firststream manager 134 for a first stream computing application owned by afirst customer and a second stream manager 134 for a second streamcomputing application owned by a second customer.

FIG. 2 is a more detailed view of the compute node 130 of FIGS. 1A-1B,according to one embodiment disclosed herein. As shown, the compute node130 includes, without limitation, at least one CPU 205, a networkinterface 215, an interconnect 220, a memory 225, and storage 230. Thecompute node 130 may also include an I/O devices interface 210 used toconnect I/O devices 212 (e.g., keyboard, display and mouse devices) tothe compute node 130.

Each CPU 205 retrieves and executes programming instructions stored inthe memory 225. Similarly, the CPU 205 stores and retrieves applicationdata residing in the memory 225. The interconnect 220 is used totransmit programming instructions and application data between each CPU205, I/O devices interface 210, storage 230, network interface 215, andmemory 225. CPU 205 is included to be representative of a single CPU,multiple CPUs, a single CPU having multiple processing cores, and thelike. In one embodiment, a PE 235 is assigned to be executed by only oneCPU 205 although in other embodiments the operators 240 of a PE 235 maycomprise one or more threads that are executed on a plurality of CPUs205. The memory 225 is generally included to be representative of arandom access memory (e.g., DRAM or Flash). Storage 230, such as a harddisk drive, solid state device (SSD), or flash memory storage drive, maystore non-volatile data.

In this example, the memory 225 includes a plurality of processingelements 235. Each PE 235 includes a collection of operators 240 thatare fused together. As noted above, each operator 240 may provide asmall chunk of code configured to process data flowing into a processingelement (e.g., PE 235) and to emit data to other operators 240 in thesame PE or to other PEs in the stream computing application. Suchprocessing elements may be on the same compute node 130 or on othercompute nodes that are accessible via communications network 120.

As shown, storage 230 contains a buffer 260. Although shown as being instorage, the buffer 260 may be located in the memory 225 of the computenode 130 or a combination of both. Moreover, storage 230 may includestorage space that is external to the compute node 130.

FIG. 3 is a more detailed view of the server management system 105 ofFIG. 1, according to one embodiment disclosed herein. As shown, servermanagement system 105 includes, without limitation, a CPU 305, a networkinterface 315, an interconnect 320, a memory 325, and storage 330. Theclient system 130 may also include an I/O device interface 310connecting I/O devices 312 (e.g., keyboard, display and mouse devices)to the server management system 105.

Like CPU 205 of FIG. 2, CPU 305 is configured to retrieve and executeprogramming instructions stored in the memory 325 and storage 330.Similarly, the CPU 305 is configured to store and retrieve applicationdata residing in the memory 325 and storage 330. The interconnect 320 isconfigured to move data, such as programming instructions andapplication data, between the CPU 305, I/O devices interface 310,storage unit 330, network interface 305, and memory 325. Like CPU 205,CPU 305 is included to be representative of a single CPU, multiple CPUs,a single CPU having multiple processing cores, and the like. Memory 325is generally included to be representative of a random access memory.The network interface 315 is configured to transmit data via thecommunications network 120. Although shown as a single unit, the storage330 may be a combination of fixed and/or removable storage devices, suchas fixed disc drives, removable memory cards, optical storage, SSD orflash memory devices, network attached storage (NAS), or connections tostorage area-network (SAN) devices.

As shown, the memory 325 stores a stream manager 134. Additionally, thestorage 330 includes a primary operator graph 132. The stream manager134 may use the primary operator graph 132 to route tuples to PEs 235for processing. The stream manager 134 also includes encryption manager333 for processing encrypted data tuples. In some embodiments, theencryption manager 333 defines properties of an operator which controlswhether the operator will decrypt encrypted data, and whether encryptedor decrypted data will be output to a downstream operator. In someembodiments, the encryption manager 333 may define the operatorproperties at compilation. In some other embodiments, the encryptionmanager 333 tags specific data tuples with attributes indicating whetherthe data should be decrypted, and whether it should be output to adownstream operator in an encrypted or decrypted format. The operatorproperties and tuple attributes may be set and stored in any mannersufficient to indicate whether an operator is permitted to decryptencrypted tuples, including, but not limited to, Boolean values andinteger values corresponding to levels of permission.

FIG. 4 is a flowchart depicting a method 400 for processing encrypteddata, according to one embodiment disclosed herein. At step 410, anoperator receives data tuples containing encrypted data attributes. Atstep 420, described in greater detail with reference to FIG. 5, theencryption manager 333 determines that a predetermined condition issatisfied such that encrypted attributes within the data tuples may bedecrypted. At step 430, described in greater detail with reference toFIG. 6, the encryption manager 333 decrypts the encrypted data. At step440, the operator performs operations on the tuples. At step 450,described in greater detail with reference to FIG. 7, the tuples aretransmitted to a downstream operator.

FIG. 5 is a flowchart depicting a method 500 corresponding to step 420for determining whether an operator may decrypt encrypted data,according to one embodiment disclosed herein. Although depicted as aflowchart, one, several, or all of the steps of the method 500 may bereferenced to determine whether the operator may decrypt encrypted data.In some embodiments, the encryption manager 333 performs the steps ofthe method 500. At each step, the encryption manager 333 determineswhether the operator's properties have been defined to allow decryptionof encrypted data. If such properties exist, the encryption manager 333may determine that a predefined condition is satisfied, and thedecryption may occur as defined in the operator properties. In someembodiments, the predefined condition may be the existence of operatorproperties permitting decryption. In other embodiments, the predefinedcondition may be attributes within a tuple setting rules for encryptedattributes. At step 510, the encryption manager 333 determines whetherthe predefined condition is satisfied based on operator propertiesspecifying that the operator may decrypt all encrypted data attributesin all tuples. In such cases, the operator properties may permitdecryption of any type of encrypted data, for any purpose. At step 520,the encryption manager 333 determines that the predefined condition issatisfied as to specific attributes in the tuple based on operatorproperties specifying that the operator may decrypt specific, enumeratedtuple attributes. For example, a medical records data tuple may includeencrypted attributes for a patient's social security number (SSN) andmedical diagnosis. The operator properties may specify that theoperatory can decrypt the SSN attribute, but not the medical diagnosisattribute. Alternatively, the operator properties may specify that theoperator may decrypt the medical diagnosis attribute, but not the SSNattribute. For example, it may not be necessary to decrypt a patient'sSSN in order to determine that the patient has a particular medicaldiagnosis. Therefore, in some embodiments, an operator property may bedefined such that the operator can decrypt specific attributes, butleave other attributes encrypted.

At step 530, the encryption manager 333 determines whether thepredefined condition is satisfied based on the presence of the encryptedtuple and a set of predefined tuples within a window of the operator. Awindow may be defined as a group of tuples received at the operator forprocessing. Thus, an operator may have a windowing condition whichexamines encrypted data tuples. The operator may be configured to onlydecrypt encrypted attributes of a set of data tuples if that set (oranother set) of data tuples are in the window at the same time. Thisallows decrypted data to be sent through the operator graph while onlyproviding access to the operators that are allowed to use it. Thus, ifthe encryption manager 333 determines that the operator has a windowingcondition which permits decryption if tuples A, B, and C are present inthe operator, and the encryption manager 333 determines that tuples A,B, and C are present in the operator, encrypted tuples may be decryptedwithin the operator. An example of such a windowing condition isprovided in FIG. 8, described in further detail below. At step 540, theencryption manager 333 determines whether the predefined condition issatisfied based on the operator decrypting data tuples to perform apredefined function. Operator properties may be set by the encryptionmanager 333 to permit decryption only to perform a specified function.Examples of such functions include, but are not limited to, decryptingdata tuples to perform a join; decrypting data tuples to performwindowing operations (or any other internal operations); decrypting datatuples to perform grouping within an aggregate function; and decryptingdata tuples to perform a sort. For example, the encryption manager 333may determine that an operator is requesting to decrypt tuples in orderto perform a join operation on tuples from two different data streams.If the encryption manager 333 determines that such decryption ispermitted in the operator properties, the predefined condition issatisfied, and the tuples may be decrypted for this limited purpose.Operators which are designed to make decisions based on encryptedattributes, such as in the join example given above, may be designed insuch a way that tuples are emitted only when a certain quantity ofresults are produced so as to maintain anonymity of the results. Thiswould prevent a case where too few inputs are fed into the operator toadequately protect the anonymous nature of the data. A threshold for aminimum number of results may be predefined by the encryption manager333, or by a user.

FIG. 6 is a flowchart depicting a method 600 corresponding to step 430for decrypting encrypted data, according to one embodiment disclosedherein. In some embodiments, the encryption manager 333 performs thesteps of the method 600. The method 600 identifies a number of differentmethods for decrypting data within an operator. Generally speaking, theencryption manager 333 may use any decryption algorithm to decryptencrypted data found in a tuple. Although depicted as a flowchart, one,several, or all of the steps of the method 600 may be referenced todetermine a method used to decrypt encrypted data within an operator. Atstep 610, the encryption manager 333 decrypts the encrypted data withinthe operator upon determining that a first condition is satisfied. Insome embodiments, the first condition is the existence of an operatorproperty specifying that the operator may decrypt the encrypted data. Atstep 620, the encryption manager 333, upon determining that a secondcondition is satisfied, decrypts the encrypted data within the operatorusing unencrypted data attributes in the tuple. In some embodiments, thesecond condition is an operator property specifying that specificencrypted tuples may be decrypted by specific tuples that containunencrypted data which may be used as input in a decryption algorithm.In such an embodiment, the encryption manager 333 uses the unencrypteddata and a general purpose decryption algorithm to decrypt an encrypteddata tuple. By operating in such a fashion, tuples may travel throughthe operator graph and be used only by the operators that have access toread them. An example would be sending medical records over the operatorgraph, but only those studies having access to them are able to extractthe data required for the study. These operators can then make abusiness decision that does not need to be encrypted from encrypteddata. At step 630, the encryption manager 333, upon determining that athird condition is satisfied, decrypts the encrypted data using a keyfound in a second tuple in a second data stream. In some embodiments,the third condition is an operator property specifying that thedecryption key may be found in a second tuple in a second data stream.In such embodiments, the decryption key flows into the operator from asecond data stream. The encryption manager 333, upon retrieving thiskey, may use it as input into the decryption algorithm internal to theoperator to decrypt the encrypted data. At step 640, the encryptionmanager 333, upon determining a fourth condition is satisfied, decryptsthe encrypted data by moving the operator and the state of the operatorto a predetermined node to perform the decryption. In some embodiments,the fourth condition may be operator properties which indicate thatdecryption (by any method) cannot be performed on the node on which theoperator currently resides. Therefore, the encryption manager 333 maymove the encrypted data to a predetermined node where the decryptionwill be performed. Alternatively, the operator and the state of theoperator may be moved to a given node to perform the decryption. Inanother embodiment, the stream manager 134 may only schedule processingelements containing operators with special decrypting attributes to runon specific nodes. In still other embodiments, operators may be fused orunfused based on encryption attributes.

FIG. 7 is a flowchart depicting a method 700 corresponding to step 450for transmitting data tuples, according to one embodiment disclosedherein. The operator-level privacy measures disclosed herein would beincomplete if an operator transmitted decrypted data to vulnerable nodesand operators. Therefore, the encryption manager 333 must determinewhether the operator can output decrypted data, whether it must outputencrypted data, or both, depending on the downstream operators. In someembodiments, the encryption manager 333 performs the steps of the method700. At step 710, the encryption manager 333 identifies the propertiesof the operator and host which are downstream from the current operator.Examples of operator and host properties include, but are not limitedto, the ability to accept encrypted data, the ability to decryptencrypted data, the ability to encrypt unencrypted data, and the abilityto transmit encrypted or unencrypted data. While the method 700 isfocused on the properties of the downstream operator and host, alternateembodiments may identify properties of the current operator and host,and whether that operator and host may transmit encrypted or unencrypteddata. At step 720, the encryption manager 333 determines whether thedownstream operator can accept unencrypted data. If the downstreamoperator can accept unencrypted data, the encryption manager 333proceeds to step 730. If the downstream operator cannot acceptunencrypted data, the encryption manager proceeds to step 750. At step730, the encryption manager 333 determines whether the operator's hostcan accept unencrypted data. If the downstream host can receiveunencrypted data, the encryption manager 333 proceeds to step 740. Ifthe downstream host cannot receive unencrypted data, the encryptionmanager proceeds to step 750. At step 740, the encryption manager maydetermine to transmit decrypted data to the downstream operator. In someembodiments, although unencrypted data may be transmitted, theencryption manager may decide to transmit encrypted data to provideadditional security. At step 750, the encryption manager 333 transmitsencrypted data to the downstream operator.

FIG. 8 illustrates a table 800 indicating whether a tuples containingencrypted data will be decrypted within an operator, according to oneembodiment described herein. The table 800 is associated with thewindowing condition described above with reference to step 530 of FIG.5. The table 800 has as row elements 810, 820, and 830, corresponding to“Tuple(s)”, “Decrypt in Operator A”, and “Decrypt in Operator B.”Therefore, based on a given set of tuples, the table will indicatewhether the windowing condition is met with respect to operators A andB, and whether the tuples will be decrypted within the operator. Thetuples in elements 850 and 860 are exemplary windows of tuples within anoperator at a given time. Therefore, element 850 indicates two tupleseach having two attributes. The first tuple has attributes F_Name(corresponding to first name) and L_Name (corresponding to last name),and the second tuple has attributes SSN and Diagnosis. Each attributehaving encrypted values is denoted by the series of *s. Element 860contains two tuples, the first tuple having F_Name and L_Nameattributes, and the second tuple having the Diagnosis attribute. Inelement 865, the table reflects that the encrypted attributes in element850 may be decrypted in operator A, because a windowing condition issatisfied. The condition may be, for example, that two tuples in thewindow have attributes containing information related to the same personfor decryption to take place. Because element 850 contains two tupleswith information which can be related to the same person, the encryptedattributes are decrypted. In element 866, the tuples in element 860 willnot be decrypted by operator A, as the windowing condition has not beenmet, since the two tuples do not have information related to the sameperson, as the diagnosis may be for any given individual. With respectto element 867, operator B will decrypt the encrypted attributes inelement 850, as the windowing condition has been satisfied. For example,the windowing condition in operator B may specify that decryption willoccur only when the F_Name, L_Name, and SSN attributes are present inthe window. As indicated in element 868, Operator B will not decrypt theattributes of the tuples in element 860, as the SSN attribute is notpresent.

Furthermore, although embodiments of the present disclosure aredescribed within the context of a stream computing application, this isnot the only context relevant to the present disclosure. Instead, such adescription is without limitation and is for illustrative purposes only.Of course, one of ordinary skill in the art will recognize thatembodiments of the present disclosure may be configured to operate withany computer system or application capable of performing the functionsdescribed herein. For example, embodiments of the disclosure may beconfigured to operate in a clustered environment with a standarddatabase processing application.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While the foregoing is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. A non-transitory computer program product,comprising: a computer-readable storage medium having computer-readableprogram code embodied therewith, the computer-readable program codecomprising: computer-readable program code configured to provide aplurality of processing elements comprising one or more operators, theoperators configured to process streaming data tuples by operation ofone or more computer processors; computer-readable program codeconfigured to define attributes of the operators, wherein the attributescomprise at least an access indicator defining processing rules fortuples containing encrypted data; computer-readable program codeconfigured to establish an operator graph of a plurality of operators,the operator graph defining at least one execution path in which a firstoperator of the plurality of operators is configured to receive datatuples from at least one upstream operator and transmit data tuples toat least one downstream operator; computer-readable program codeconfigured to, upon receiving a first data stream having a first tuplecontaining encrypted data in the first operator, determine, based on theaccess indicator of the first operator, whether to decrypt the encrypteddata in the first operator; computer-readable program code configuredto, upon determining that the access indicator of the first operatorpermits decryption of the encrypted data based on a windowing conditionexisting within the first operator, decrypt the encrypted data, whereinthe windowing condition is defined by the presence of both the firsttuple and a predefined set of tuples within a window of the firstoperator during a predefined period of time; and computer-readableprogram code configured to transmit the first tuple to a secondoperator, downstream from the first operator.
 2. The non-transitorycomputer program product of claim 1, wherein the processing rules of theaccess indicator further comprise at least one of: ignoring, in theoperator, tuples that contain encrypted data, wherein ignoring the tupleindicates that the encrypted data is not decrypted; ignoring, in theoperator, tuples that contain encrypted data in predefined attributes,wherein ignoring the tuple indicates that the encrypted data in thepredefined attributes is not decrypted; decrypting, in the operator,encrypted data within a tuple; and decrypting, in the operator,encrypted data for a limited purpose.
 3. The non-transitory computerprogram product of claim 2, wherein the limited purpose comprises atleast one of: decrypting to perform a join operation on at least a firsttuple and a second tuple; decrypting to perform a windowing operation;decrypting to perform grouping within an aggregate function; anddecrypting to perform a sort.
 4. The non-transitory computer programproduct of claim 1, wherein decrypting the encrypted data comprises atleast one of: decrypting the encrypted data within the operator;decrypting the encrypted data by moving the operator and the state ofthe operator to a predetermined node to perform the decryption;decrypting the encrypted data using a second data in the first tuple;and decrypting the encrypted data using a key from a second tuple in asecond data stream.
 5. The non-transitory computer program product ofclaim 1, wherein transmitting the first tuple comprises: upondetermining the access indicator of the second operator does not permitthe second operator to receive unencrypted data, transmitting theencrypted data to the second operator; upon determining the accessindicator of the second operator permits the second operator to receiveunencrypted data, transmitting the decrypted data to the secondoperator; upon determining that a computing system hosting theprocessing element containing the second operator is not permitted toreceive unencrypted data, transmitting the encrypted data to the secondoperator; and upon determining that a computing system hosting theprocessing element containing the second operator is permitted toreceive unencrypted data, transmitting the decrypted data to the secondoperator.
 6. The non-transitory computer program product of claim 1,wherein the first tuple is transmitted upon determining that the numberof tuples received in the first operator exceed a predefined threshold.7. A system, comprising: one or more computer processors; and a memorycontaining a program, which when executed by the one or more computerprocessors is configured to perform an operation, the operationcomprising: providing a plurality of processing elements comprising oneor more operators, the operators configured to process streaming datatuples by operation of one or more computer processors; definingattributes of the operators, wherein the attributes comprise at least anaccess indicator defining processing rules for tuples containingencrypted data; establishing an operator graph of a plurality ofoperators, the operator graph defining at least one execution path inwhich a first operator of the plurality of operators is configured toreceive data tuples from at least one upstream operator and transmitdata tuples to at least one downstream operator; upon receiving a firstdata stream having a first tuple containing encrypted data in the firstoperator, determining, based on the access indicator of the firstoperator, whether to decrypt the encrypted data in the first operator;upon determining the access indicator of the first operator permitsdecryption of the encrypted data, decrypting the encrypted data; andtransmitting the first tuple to a second operator, downstream from thefirst operator, wherein transmitting the first tuple comprises: upondetermining that a computing system hosting the processing elementcontaining the second operator is not permitted to receive unencrypteddata, transmitting the encrypted data to the second operator; and upondetermining that a computing system hosting the processing elementcontaining the second operator is permitted to receive unencrypted data,transmitting the decrypted data to the second operator.
 8. The system ofclaim 7, wherein the processing rules of the access indicator compriseat least one of: ignoring, in the operator, tuples that containencrypted data, wherein ignoring the tuple indicates that the encrypteddata is not decrypted; ignoring, in the operator, tuples that containencrypted data in predefined attributes, wherein ignoring the tupleindicates that the encrypted data in the predefined attributes is notdecrypted; decrypting, in the operator, encrypted data within a tuple;upon determining a windowing condition exists within the operator,decrypting, in the operator, encrypted data within a tuple; anddecrypting, in the operator, encrypted data for a limited purpose. 9.The system of claim 8, wherein the windowing condition is defined by thepresence of both the encrypted tuple and a predefined set of tupleswithin a window of the operator during a predefined period of time. 10.The system of claim 8, wherein the limited purpose comprises at leastone of: decrypting to perform a join operation on at least a first tupleand a second tuple; decrypting to perform a windowing operation;decrypting to perform grouping within an aggregate function; anddecrypting to perform a sort.
 11. The system of claim 7, whereindecrypting the encrypted data comprises at least one of: decrypting theencrypted data within the operator; decrypting the encrypted data bymoving the operator and the state of the operator to a predeterminednode to perform the decryption; decrypting the encrypted data using asecond data in the first tuple; and decrypting the encrypted data usinga key from a second tuple in a second data stream.
 12. The system ofclaim 7, wherein transmitting the first tuple further comprises: upondetermining the access indicator of the second operator does not permitthe second operator to receive unencrypted data, transmitting theencrypted data to the second operator; and upon determining the accessindicator of the second operator permits the second operator to receiveunencrypted data, transmitting the decrypted data to the secondoperator.
 13. The system of claim 7, wherein the first tuple istransmitted upon determining that the number of tuples received in thefirst operator exceed a predefined threshold.
 14. A non-transitorycomputer program product, comprising: a computer-readable storage mediumhaving computer-readable program code embodied therewith, thecomputer-readable program code comprising: computer-readable programcode configured to provide a plurality of processing elements comprisingone or more operators, the operators configured to process streamingdata tuples by operation of one or more computer processors;computer-readable program code configured to define attributes of theoperators, wherein the attributes comprise at least an access indicatordefining processing rules for tuples containing encrypted data;computer-readable program code configured to establish an operator graphof a plurality of operators, the operator graph defining at least oneexecution path in which a first operator of the plurality of operatorsis configured to receive data tuples from at least one upstream operatorand transmit data tuples to at least one downstream operator;computer-readable program code configured to, upon receiving a firstdata stream having a first tuple containing encrypted data in the firstoperator, determine, based on the access indicator of the firstoperator, whether to decrypt the encrypted data in the first operator;computer-readable program code configured to, upon determining theaccess indicator of the first operator permits decryption of theencrypted data, decrypt the encrypted data; and computer-readableprogram code configured to transmit the first tuple to a secondoperator, downstream from the first operator, wherein transmitting thefirst tuple comprises: upon determining the access indicator of thesecond operator does not permit the second operator to receiveunencrypted data, transmitting the encrypted data to the secondoperator; and upon determining the access indicator of the secondoperator permits the second operator to receive unencrypted data,transmitting the decrypted data to the second operator.